Individually Conditional Individual Mutual Information Bound on Generalization Error

Mutual Information and Conditional Mean Prediction Error

Mutual information is fundamentally important for measuring statistical dependence between variables and for quantifying information transfer by signaling and communication mechanisms. It can, however, be challenging to evaluate for physical models of such mechanisms and to estimate reliably from data. Furthermore, its relationship to better known statistical procedures is still poorly understo...

The Intrinsic Conditional Mutual Information

This paper is concerned with secret key agreement by public discussion: two parties Alice and Bob and an adversary Eve have access to independent realizations of random variables X, Y , and Z, respectively, with joint distribution P XY Z. The secret key rate S(X; Y jjZ) has been deened as the maximal rate at which Alice and Bob can generate a secret key by communication over an insecure, but au...

About the Mutual (Conditional) Information

Directed Information and Conditional Mutual Information

—We study directed information in Bayesian networks and related structures. Mutual information is split into directed information and residual information. Some basic equations for directed information and residual information are determined.

The Intrinsic Conditional Mutual Information andPerfect

This paper is concerned with secret key agreement by public discussion: two parties Alice and Bob and an adversary Eve have access to independent realizations of random variables X, Y , and Z, respectively, with joint distribution P XY Z. The secret key rate S(X; Y jjZ) has been deened as the maximal rate at which Alice and Bob can generate a secret key by communication over an insecure, but au...